{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f7FPLsj4nB-6"
   },
   "source": [
    "#  Modeling Protein-Ligand Interactions with Atomic Convolutions\n",
    "By [Nathan C. Frey](https://ncfrey.github.io/) | [Twitter](https://twitter.com/nc_frey) and [Bharath Ramsundar](https://rbharath.github.io/) | [Twitter](https://twitter.com/rbhar90)\n",
    "\n",
    "This DeepChem tutorial introduces the [Atomic Convolutional Neural Network](https://arxiv.org/pdf/1703.10603.pdf). We'll see the structure of the `AtomicConvModel` and write a simple program to run Atomic Convolutions.\n",
    "\n",
    "### ACNN Architecture\n",
    "ACNN’s directly exploit the local three-dimensional structure of molecules to hierarchically learn more complex chemical features by optimizing both the model and featurization simultaneously in an end-to-end fashion.\n",
    "\n",
    "The atom type convolution makes use of a neighbor-listed distance matrix to extract features encoding local chemical environments from an input representation (Cartesian atomic coordinates) that does not necessarily contain spatial locality. The following methods are used to build the ACNN architecture:\n",
    "\n",
    "- __Distance Matrix__  \n",
    "The distance matrix $R$ is constructed from the Cartesian atomic coordinates $X$. It calculates distances from the distance tensor $D$. The distance matrix construction accepts as input a $(N, 3)$ coordinate matrix $C$. This matrix is “neighbor listed” into a $(N, M)$ matrix $R$.\n",
    "\n",
    "```python\n",
    "    R = tf.reduce_sum(tf.multiply(D, D), 3)     # D: Distance Tensor\n",
    "    R = tf.sqrt(R)                              # R: Distance Matrix\n",
    "    return R\n",
    "```\n",
    "\n",
    "- **Atom type convolution**  \n",
    "The output of the atom type convolution is constructed from the distance matrix $R$ and atomic number matrix $Z$. The matrix $R$ is fed into a (1x1) filter with stride 1 and depth of $N_{at}$ , where $N_{at}$ is the number of unique atomic numbers (atom types) present in the molecular system. The atom type convolution kernel is a step function that operates on the neighbor distance matrix $R$.\n",
    "\n",
    "- **Radial Pooling layer**  \n",
    "Radial Pooling is basically a dimensionality reduction process that down-samples the output of the atom type convolutions. The reduction process prevents overfitting by providing an abstracted form of representation through feature binning, as well as reducing the number of parameters learned.\n",
    "Mathematically, radial pooling layers pool over tensor slices (receptive fields) of size (1x$M$x1) with stride 1 and a depth of $N_r$, where $N_r$ is the number of desired radial filters and $M$ is the maximum number of neighbors.\n",
    "\n",
    "- **Atomistic fully connected network**  \n",
    "Atomic Convolution layers are stacked by feeding the flattened ($N$, $N_{at}$ $\\cdot$ $N_r$) output of the radial pooling layer into the atom type convolution operation. Finally, we feed the tensor row-wise (per-atom) into a fully-connected network. The\n",
    "same fully connected weights and biases are used for each atom in a given molecule.\n",
    "\n",
    "Now that we have seen the structural overview of ACNNs, we'll try to get deeper into the model and see how we can train it and what we expect as the output.\n",
    "\n",
    "For the training, we will use the publicly available PDBbind dataset. In this example, every row reflects a protein-ligand complex and the target is the binding affinity ($K_i$) of the ligand to the protein in the complex.\n",
    "\n",
    "## Colab\n",
    "\n",
    "This tutorial and the rest in this sequence are designed to be done in Google colab. If you'd like to open this notebook in colab, you can use the following link.\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/deepchem/deepchem/blob/master/examples/tutorials/Modeling_Protein_Ligand_Interactions_With_Atomic_Convolutions.ipynb)\n",
    "\n",
    "## Setup\n",
    "\n",
    "To run DeepChem within Colab, you'll need to run the following cell of installation commands. This will take about 5 minutes to run to completion and install your environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Y2xCQyOInB_D"
   },
   "outputs": [],
   "source": [
    "!pip install -q condacolab\n",
    "import condacolab\n",
    "condacolab.install()\n",
    "!/usr/local/bin/conda info -e"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "WKxOGlhhMrC7"
   },
   "outputs": [],
   "source": [
    "!/usr/local/bin/conda install -c conda-forge pycosat mdtraj pdbfixer openmm -y -q  # needed for AtomicConvs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jFQmra_fFE8U"
   },
   "outputs": [],
   "source": [
    "!pip install --pre deepchem\n",
    "import deepchem\n",
    "deepchem.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "W1cCOOYXnB_L"
   },
   "outputs": [],
   "source": [
    "import deepchem as dc\n",
    "import os\n",
    "\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from rdkit import Chem\n",
    "\n",
    "from deepchem.molnet import load_pdbbind\n",
    "from deepchem.models import AtomicConvModel\n",
    "from deepchem.feat import AtomicConvFeaturizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fACLMson_-vk"
   },
   "source": [
    "### Getting protein-ligand data\n",
    "If you worked through [Tutorial 13](https://github.com/deepchem/deepchem/blob/master/examples/tutorials/Modeling_Protein_Ligand_Interactions.ipynb) on modeling protein-ligand interactions, you'll already be familiar with how to obtain a set of data from PDBbind for training our model. Since we explored molecular complexes in detail in the [previous tutorial]((https://github.com/deepchem/deepchem/blob/master/examples/tutorials/Modeling_Protein_Ligand_Interactions.ipynb)), this time we'll simply initialize an `AtomicConvFeaturizer` and load the PDBbind dataset directly using MolNet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_qu5DlVa3aV3"
   },
   "outputs": [],
   "source": [
    "f1_num_atoms = 100  # maximum number of atoms to consider in the ligand\n",
    "f2_num_atoms = 1000  # maximum number of atoms to consider in the protein\n",
    "max_num_neighbors = 12  # maximum number of spatial neighbors for an atom\n",
    "\n",
    "acf = AtomicConvFeaturizer(frag1_num_atoms=f1_num_atoms,\n",
    "                      frag2_num_atoms=f2_num_atoms,\n",
    "                      complex_num_atoms=f1_num_atoms+f2_num_atoms,\n",
    "                      max_num_neighbors=max_num_neighbors,\n",
    "                      neighbor_cutoff=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pyH9KUkvxlxk"
   },
   "source": [
    "`load_pdbbind` allows us to specify if we want to use the entire protein or only the binding pocket (`pocket=True`) for featurization. Using only the pocket saves memory and speeds up the featurization. We can also use the \"core\" dataset of ~200 high-quality complexes for rapidly testing our model, or the larger \"refined\" set of nearly 5000 complexes for more datapoints and more robust training/validation. On Colab, it takes only a minute to featurize the core PDBbind set! This is pretty incredible, and it means you can quickly experiment with different featurizations and model architectures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Z9eyanh35qyj",
    "outputId": "1bdc22d2-bf73-48cc-9f31-ecc0f56cf4bc"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.7/dist-packages/numpy/core/_asarray.py:83: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray\n",
      "  return array(a, dtype, copy=False, order=order)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 43.2 s, sys: 18.6 s, total: 1min 1s\n",
      "Wall time: 1min 10s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "tasks, datasets, transformers = load_pdbbind(featurizer=acf,\n",
    "                                             save_dir='.',\n",
    "                                             data_dir='.',\n",
    "                                             pocket=True,\n",
    "                                             reload=False,\n",
    "                                             set_name='core')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Unfortunately, if you try to use the \"refined\" dataset, there are some complexes that cannot be featurized. To resolve this issue, rather than increasing `complex_num_atoms`, simply omit the lines of the dataset that have an `x` value of `None`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyTransformer(dc.trans.Transformer):\n",
    "  def transform_array(x, y, w, ids):\n",
    "    kept_rows = x != None\n",
    "    return x[kept_rows], y[kept_rows], w[kept_rows], ids[kept_rows],\n",
    "\n",
    "datasets = [d.transform(MyTransformer) for d in datasets]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "EaGn9UbwEdyY",
    "outputId": "6b235d21-88a2-45b0-d6ba-55a4d6a72dd9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<DiskDataset X.shape: (154, 9), y.shape: (154,), w.shape: (154,), ids: ['1mq6' '3pe2' '2wtv' ... '3f3c' '4gqq' '2x00'], task_names: [0]>,\n",
       " <DiskDataset X.shape: (19, 9), y.shape: (19,), w.shape: (19,), ids: ['3ivg' '4de1' '4tmn' ... '2vw5' '1w3l' '2zjw'], task_names: [0]>,\n",
       " <DiskDataset X.shape: (20, 9), y.shape: (20,), w.shape: (20,), ids: ['1kel' '2w66' '2xnb' ... '2qbp' '3lka' '1qi0'], task_names: [0]>)"
      ]
     },
     "execution_count": 21,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PQq0lkWIfVoE"
   },
   "outputs": [],
   "source": [
    "train, val, test = datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GNilV3VXnB_j"
   },
   "source": [
    "### Training the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WufupHBPnB_k"
   },
   "source": [
    "Now that we've got our dataset, let's go ahead and initialize an `AtomicConvModel` to train. Keep the input parameters the same as those used in `AtomicConvFeaturizer`, or else we'll get errors. `layer_sizes` controls the number of layers and the size of each dense layer in the network. We choose these hyperparameters to be the same as those used in the [original paper](https://arxiv.org/pdf/1703.10603.pdf)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ErBNNGH55-_B"
   },
   "outputs": [],
   "source": [
    "acm = AtomicConvModel(n_tasks=1,\n",
    "                      frag1_num_atoms=f1_num_atoms,\n",
    "                      frag2_num_atoms=f2_num_atoms,\n",
    "                      complex_num_atoms=f1_num_atoms+f2_num_atoms,\n",
    "                      max_num_neighbors=max_num_neighbors,\n",
    "                      batch_size=12,\n",
    "                      layer_sizes=[32, 32, 16],\n",
    "                      learning_rate=0.003,\n",
    "                      )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "4cNdP1b1hEQM"
   },
   "outputs": [],
   "source": [
    "losses, val_losses = [], []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5g6b2qEwNwdL",
    "outputId": "3caa11ac-18dd-4528-f966-dee61d2c508d"
   },
"outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 2min 41s, sys: 11.4 s, total: 2min 53s\n",
            "Wall time: 2min 47s\n"
          ]
        }
      ],
   "source": [
    "%%time\n",
    "max_epochs = 50\n",
    "\n",
    "metric = dc.metrics.Metric(dc.metrics.score_function.rms_score)\n",
    "step_cutoff = len(train)//12\n",
    "def val_cb(model, step):\n",
    "  if step%step_cutoff!=0:\n",
    "      return\n",
    "  val_losses.append(model.evaluate(val, metrics=[metric])['rms_score']**2)  # L2 Loss\n",
    "  losses.append(model.evaluate(train, metrics=[metric])['rms_score']**2)  # L2 Loss\n",
    "\n",
    "acm.fit(train, nb_epoch=max_epochs, max_checkpoints_to_keep=1,\n",
    "                callbacks=[val_cb])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aTFdba_KDDUQ"
   },
   "source": [
    "The loss curves are not exactly smooth, which is unsurprising because we are using 154 training and 19 validation datapoints. Increasing the dataset size may help with this, but will also require greater computational resources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 265
    },
    "id": "pn4QWM1bizw0",
    "outputId": "f6399595-4622-41d0-80d1-92df68850cc2"
   },
 "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5RcVb3g8e+PpiENhHQMEaETSJjBAHmQQAfjyuVlkASUEFAExBlx8VjOoKLeyRCuArlcZxGNI9esAVm5DAiIEkSIuYLGKwQDXvDSefEIRCA80g3XdCKdAdNcOslv/qjqpLpSj12nzqmz9+nfZ61e3XXqdNU+Vef8zt6/vfc5oqoYY4wJ3z5pF8AYY0w8LKAbY0xGWEA3xpiMsIBujDEZYQHdGGMyYt+03viQQw7RMWPGpPX2xhgTpFWrVm1R1ZGlnkstoI8ZM4aOjo603t4YY4IkIm+Ue85SLsYYkxEW0I0xJiMsoBtjTEaklkM3xmRXX18fnZ2dvP/++2kXJVhDhgxh1KhRNDc3O/+PBXRjTOw6OzsZOnQoY8aMQUTSLk5wVJWtW7fS2dnJ2LFjnf/PUi6+efZ+uHkCzG/N/X72/rRLZEzN3n//fUaMGGHBPCIRYcSIETW3cKyG7pNn74d//hr09eYeb9uUewww6XPplcuYCCyY1yfK52c1dJ88euOeYN6vrze33BhjqrCA7pNtnbUtN8aU1NPTw6233hrpf88++2x6enqc158/fz7f//73I71X3Cyg+2TYqNqWG2NKqhTQd+zYUfF/H3nkEVpbW5MoVuKqBnQRuUNENovI81XWmyoiO0Tks/EVb5CZcT00twxc1tySW25Mhi1d08X0BY8xdt7DTF/wGEvXdNX1evPmzePVV19l8uTJzJ07l8cff5yTTz6Z2bNnc9xxxwEwZ84cTjzxRMaPH8/ixYt3/++YMWPYsmULr7/+OsceeyxXXHEF48eP58wzz6S3t7fcWwKwdu1apk2bxqRJkzjvvPN45513AFi0aBHHHXcckyZN4qKLLgLg97//PZMnT2by5MlMmTKFd999t65tBnLDYyr9AKcAJwDPV1inCXgMeAT4bLXXVFVOPPFENSWsW6L6g/GqNwzL/V63JO0SGVOz9evXO6/70OpOPebbv9Yjr/nV7p9jvv1rfWh1Z+T3f+2113T8+PG7H69YsUIPOOAA3bhx4+5lW7duVVXV7du36/jx43XLli2qqnrkkUdqd3e3vvbaa9rU1KRr1qxRVdULLrhA77nnnr3e64YbbtCFCxeqqurEiRP18ccfV1XV6667Tq+++mpVVT3ssMP0/fffV1XVd955R1VVP/3pT+uTTz6pqqrvvvuu9vX17fXapT5HoEPLxNWqNXRVXQn8pcpqXwV+AWyOfmoxQG40yzeeh/k9ud82usVk3MLlG+jt2zlgWW/fThYu3xDr+5x00kkDxnQvWrSI448/nmnTprFp0yZefvnlvf5n7NixTJ48GYATTzyR119/vezrb9u2jZ6eHk499VQAvvjFL7Jy5UoAJk2axCWXXMJPfvIT9t03N7hw+vTpfPOb32TRokX09PTsXl6PunPoItIGnAf8yGHdK0WkQ0Q6uru7631rY0wGvNVTOo1RbnlUBx544O6/H3/8cX73u9/x1FNPsW7dOqZMmVJyzPf++++/+++mpqaq+fdyHn74Ya666ipWr17N1KlT2bFjB/PmzeP222+nt7eX6dOn89JLL0V67UJxdIr+I3CNqu6qtqKqLlbVdlVtHzmy5OV8jTGDzOGtLTUtdzF06NCKOelt27YxfPhwDjjgAF566SWefvrpyO/Vb9iwYQwfPpwnnngCgHvuuYdTTz2VXbt2sWnTJk4//XS++93vsm3bNt577z1effVVJk6cyDXXXMPUqVNjCehxTCxqB+7LD4I/BDhbRHao6tIYXtsYk3FzZ47j2gefG5B2aWluYu7McZFfc8SIEUyfPp0JEyZw1lln8alPfWrA87NmzeK2227j2GOPZdy4cUybNi3yexW66667+PKXv8z27ds56qijuPPOO9m5cydf+MIX2LZtG6rK1772NVpbW7nuuutYsWIF++yzD+PHj+ess86q+/0ll2OvspLIGOBXqjqhyno/zq/3QLXXbG9vV7vBhTHZ9OKLL3Lsscc6r790TRcLl2/grZ5eDm9tYe7MccyZ0pZgCcNQ6nMUkVWq2l5q/ao1dBH5GXAacIiIdAI3AM0AqnpbvQU2xpg5U9osgMegakBX1YtdX0xVL62rNMYYYyKzmaLGGJMRFtCNMSYjLKAbY0xGWEA3xpiMsIBujDHAQQcdVNNyH1lAN8aYjLCAboxJX8z30p03bx633HLL7sf9N6F47733mDFjBieccAITJ07kl7/8pfNrqipz585lwoQJTJw4kSVLlgDw9ttvc8oppzB58mQmTJjAE088wc6dO7n00kt3r3vzzTfXtT2u7J6ixph0JXAv3QsvvJCvf/3rXHXVVQDcf//9LF++nCFDhvDQQw9x8MEHs2XLFqZNm8bs2bOd7t/54IMPsnbtWtatW8eWLVuYOnUqp5xyCj/96U+ZOXMm3/rWt9i5cyfbt29n7dq1dHV18fzzudtI1HIHpHpYDd0Yk64E7qU7ZcoUNm/ezFtvvcW6desYPnw4o0ePRlX5u7/7OyZNmsQZZ5xBV1cXf/7zn51e88knn+Tiiy+mqamJQw89lFNPPZVnnnmGqVOncueddzJ//nyee+45hg4dylFHHcXGjRv56le/ym9+8xsOPvjgyNtSCwvoxoQs5lRFKhK6l+4FF1zAAw88wJIlS7jwwgsBuPfee+nu7mbVqlWsXbuWQw89tORlc2txyimnsHLlStra2rj00ku5++67GT58OOvWreO0007jtttu4/LLL6/rPVxZQA9RFg5iU7/+VMW2TYDuSVWEtj8kdC/dCy+8kPvuu48HHniACy64AMhdNvfDH/4wzc3NrFixgjfeeMP59U4++WSWLFnCzp076e7uZuXKlZx00km88cYbHHrooVxxxRVcfvnlrF69mi1btrBr1y4+85nP8J3vfIfVq1fXtS2uLIcemgTyjSZQlVIVIe0LM64fuE9DLPfSHT9+PO+++y5tbW0cdthhAFxyySWcc845TJw4kfb2do455hjn1zvvvPN46qmnOP744xERvve97/GRj3yEu+66i4ULF9Lc3MxBBx3E3XffTVdXF1/60pfYtSt3m4ibbrqprm1x5XT53CTY5XMjunlCvkZWZNjo3C3rzOAxvxUodfxK7haGKar18rk8e3/uRLStM1czn3F9WCelhMR++VzjmYTyjSZAw0aVObnXl6pIxaTPWQCPgeXQQ5NQvtEEaMb1udREoRhSFSZcFtBDYwex6Tfpc3DOoly6Dcn9PmeRNzXdtNK5WRHl87OUS2j6D1bLNxrwNlUxZMgQtm7dyogRI5wm7ZiBVJWtW7cyZMiQmv7PAnqIPD2Ijek3atQoOjs76e7uTrsowRoyZAijRtWWSrWAboyJXXNzM2PHjk27GIOO5dCNMSYjLKAbY0xGVA3oInKHiGwWkZKzVkTkEhF5VkSeE5F/FZHj4y+mMcaYalxq6D8GZlV4/jXgVFWdCPwDsDiGchljjKlR1U5RVV0pImMqPP+vBQ+fBmyGizHGpCDuHPplwK/LPSkiV4pIh4h02HAmY4yJV2wBXUROJxfQrym3jqouVtV2VW0fOXJkXG9tjDGGmMahi8gk4HbgLFXdGsdrGmOMqU3dNXQROQJ4EPgvqvqn+otkjDEmiqo1dBH5GXAacIiIdAI3AM0AqnobcD0wArg1f82GHeWu1WuMMSY5LqNcLq7y/OVAY26YZ4wxpiybKWqMCYvdU7csuziXMSYcdk/diqyGbowJR6UbYxsL6MaYgNg9dSuygJ4mywUaUxu7p25FFtDT0p8L3LYJ0D25QAvqxpRn99StyAJ6WiwXaEztPL8xdtpslEtaLBdoTDR2T92yrIaeFssFmiyz/qFUWEBPi+UCTVZZ/1BqLKCnxXKBJqusfyg12cyhP3t/bufZ1plLYcy43s9AablAk0XWP5SasGroLnk5a+6lw3Kmpp/1D6UmnIDuGqitudd4dhI1hax/KDXhBHTXQB1Qc2/pmi6mL3iMsfMeZvqCx1i6pivtIkVjJ1FTqJ7+IWvp1SWcHLproB42Kl9TZO/lHlm6potrH3yO3r6dAHT19HLtg88BMGdKW+MKEkd/Q0AnUZOXdD9TlP4hu5Ji3cKpobvm5QJp7i1cvmF3MO/X27eThcs3NK4QrqmSarUmy5mGxdcUmbX06hZOQHcN1EkPB4ypSfhWT29NyxPhcgC5HPyBnERNnq+B01p6dQsn5dIfkF2aiUkNB4yxSXh4awtdJYL34a0tJdZOiMsBVOng79/mWr4bkz5fA2cg6VKfhRPQIf1x2y7BzdHcmeMG5NABWpqbmDtz3MAVk8x1uhxArgd/2t+Ncedr4Jxx/cAKE1hLr0bhpFx8EGPNZs6UNm46fyJtrS0I0Nbawk3nTxzYIZp0rtMlVWL58fBUSwv6miKz2dN1q1pDF5E7gE8Dm1V1QonnBfghcDawHbhUVVfHXVAvxFyzmTOlrfKIlhhbBCW5pEqs1hQWl7Sgzykya+nVxSXl8mPg/wB3l3n+LODo/M/HgB/lf2dPo4NbI3Kd1Q4gnw9+szfXSoAFznh5crmRqgFdVVeKyJgKq5wL3K2qCjwtIq0icpiqvh1TGf3R6ODmS67TDv5w+NrhmWUejZ+Po1O0DSiMOp35ZdkL6NDY4GbpDlMrXyoBPkqqFp10arQGDe0UFZErRaRDRDq6u7sb+dZhiruTKGvTqrO2PXGIs8MzS59vkgMMPGoVxVFD7wJGFzwelV+2F1VdDCwGaG9v1xjeO/viahF41CyMRRrb40metKK40oJZ21+SrEV71CqKo4a+DPivkjMN2JbJ/HnofJ0dGFWjtyeN6fJRa8iTPgffeB7m9+R+RwlYWdtfkqxFezQMtGpAF5GfAU8B40SkU0QuE5Evi8iX86s8AmwEXgH+CfjviZXWRBf3Dp12czzO7XHZlsFwAinkURohFknOp/Bo/LzLKJeLqzyvwFWxlcgkI85moQ/N8bi2x3VbGh3g0u5o8yiNEIt6Bhi4pNo8GQlmM0WTknYNtliczUIfmuOu21Pte3DdlkbPmE27hlzu8z36TL/2a1dRa9Fpt5RqZAE9CT7uBHE2C9MINsWBGapvj8v34Lotjc6Tpn3JhVL7y/Gfh3U/jbZf+1DBKdW3ENcJ3xNhXZwrFGk3l8uJq1nY6OZ4ubTIOYtyB2Y5Lt+D67Y0elKZD3MQiveXmydE2699SNFFLVfaLaUaWQ09CYHtBDVr9FjnqLUkl++hlm2JOnokSu3Uo4623aLu1/XUcpOs2buUK+2WUo2shp6ErHUoFWv0WOeogcTle0i65l1P7dSTjrbdou7XUb+/pGv2rif8tFtKNbAaehI8GpeamEaOdY5aS6rlLlf1bks5PuRg46rlRt2vo35/SX92LuXysaVUgdXQk5DGFQqLh1YdfSa8/Nt0ZzVWG+5VS4dklFpSI2rfxa9d/H6larTQuPRbnLXcqJ9n1O+vlpp9lFm8ruXyraVUgeSGkTdee3u7dnR0pPLemVN80JbS3NLYmkWpMhWX4eYJpQNey4dgvwMrB8q0p92X2r59mkEEdn5QsKIAJY6xYaMrd+jGpdxn3Kj37xcl4LqW3WVfi7NcKRORVaraXvI5C+gZUG7HL9bIg9jlYHQNio0+Gblw/cyBvYJ6I7dnfislTyhILsVUyLfg5hqofTlpNUilgG459Cxwbb43cpSNS3O5VH5y/6FFNVz8HPdb02epe4/nfvTGxozJds1f1zN3IqmRKK7566yPKquB5dCzoFKutni9RqllfHfhATq/tfTr+XZwun7mULlV0h8433w6mT4P1zxx1LkTSY9EcclfZ31UWQ3Cr6H7MAMtbaVGHxRr9CibRo+IaLRS27dPMzTtN3BZ8TaXC5wddyQzszjpWq4Po3gGw6gyR2EHdNdmYtaDfqmDtv2ydIdaRR3uFcrBWWr75twK595SeZvLBsiiPHecQdFlWGbUE6kP6Y44hxYGHivC7hSN2vFWrlMqaqeQb51Jocvy51lrZ2pxx2VSoo4UyVKHZD2jZRoou52iLrUD1yZh1E4hHy/EFbokJ/qkrWR6TEqv28g0U9ZbVC58SB/VKexOUZfOENcmYdROIdf/i1jrXLqmi4XLN/BWTy+Ht7Ywd+Y45kxpq/p/xlOlJuccfWbuKoZpTy+PMoEmjUl0SfEhfVSnsAO6Sw9+uaDfMjzfXKxzRp/LThBxJMDSNV1c++Bz9PbtBKCrp5drH3wOwIJ6yEoFziOmhRsUA5pJWVEGRsuEnXJxaSaWG43wwXsD0yRRm70unUkRm3ILl2/YHcz79fbtZOHyDZXL5LPAO50Sk+U0Uzm+7QuNSB8lvM1h19Cheu2gVJPwg79C71+KVlRKzujrv0NLuZqTSyshYlPurZ7SU/nLLfeer9fFLiXLHbM+8HFf8PnKm47CD+guXCev9M/oK5fbLPUFuOwEEZtyh7e20FUieB/eWmXMua98vfFHMR+DTdb4ui+4po+inPAbsM2DI6AXKxtgi4Zaud6hpdpOEPFqc3NnjhuQQwdoaW5i7sxxFf/PW6F0OvkabLIklH2hFNcTfnHQb8CVN8POoUflmiuLa6eLOCRszpQ2bjp/Im2tLQjQ1trCTedPDLdDNJRZoCEHm1CEsi+U4tInVmo4cwOGpzrV0EVkFvBDoAm4XVUXFD1/BHAX0JpfZ56qPhJbKePmmiuLs9c74kiAOVPawg3gxUK5+0sGRjt4L+l9Ick+kKjzX8r108W4/1etoYtIE3ALcBZwHHCxiBxXtNq3gftVdQpwEXBrbCVMisuogixNmvBB0nd/SfvOPMZdkvtC0pP9XFoXlS7xkOAlOVxq6CcBr6jqRgARuQ84F1g/sJQcnP97GPBWbCVMU5YmTfgiqTHLPtyZx9QmqX0h6T6Qeua/JHxJBJeA3gYUlqwT+FjROvOB34rIV4EDgTNKvZCIXAlcCXDEEUfUWtZ0ZGXSRNbVchC7NMftew9X0n0gLif8lNKLcY1yuRj4sar+bxH5OHCPiExQ1V2FK6nqYmAx5C7OVeub2DR4U5brQRzykEQbG++mEX0gUea/NOD7cgnoXcDogsej8ssKXQbMAlDVp0RkCHAIsDmOQoJNgzdVuB7EoQ5JDPlE1Gi+dL6n0MpzGbb4DHC0iIwVkf3IdXouK1rnTWAGgIgcCwwBuuMsaCanwZv4NHooaqNl4EqADZN057vHqtbQVXWHiHwFWE5uSOIdqvqCiNwIdKjqMuBvgX8SkW+Q6yC9VGO+0HrmpsGbeKUxFLWRQj0RpWWQ9oE45dDzY8ofKVp2fcHf64Hp8RZtoMxNgzfxczmIfWmO1yrUE5FpqGBmis6dOY6W5qYBy4KeBm/SEWpz3MbGGwfBXMulv+PTRrmYuoXYHLex8cZB2PcUNfGzoXHGeK3SPUWDqaGbBrChccYELZgcumkAGxpnTNAsoJs9bGicMUGzgG72CPka1cYYC+imgA2NMyZoFtDNHqGO0TbGADbKxRQLcYy2MQbIaEC3y+waYwajzAX00C+zaycjY0xUmcuhh3yZ3f6TUVdPL8qek9HSNcWXnzfGmL1lLqCHfJndkE9Gxpj0ZS6gl7ucbgiX2Q35ZGSMSV/mAnrIl9kN+WRkjElf5gL6nClt3HT+RNpaWxCgrbWFm86fGETHYsgnI2NM+jI3ygVyQT2EAF7MrvlujKlH8AE9a8P8Qj0ZGWPSF3RAD33MuTHGxCnoHLoN8zPGmD2cArqIzBKRDSLyiojMK7PO50RkvYi8ICI/jbeYpdkwP2OM2aNqykVEmoBbgE8CncAzIrJMVdcXrHM0cC0wXVXfEZEPJ1XgQoe3ttBVInjbMD9jzGDkUkM/CXhFVTeq6gfAfcC5RetcAdyiqu8AqOrmeItZmg3zM8aYPVwCehuwqeBxZ35ZoY8CHxWRP4jI0yIyq9QLiciVItIhIh3d3d3RSlwg5DHnxhgTt7hGuewLHA2cBowCVorIRFXtKVxJVRcDiwHa29s1jje2YX7GGJPjUkPvAkYXPB6VX1aoE1imqn2q+hrwJ3IB3hhjTIO4BPRngKNFZKyI7AdcBCwrWmcpudo5InIIuRTMxhjLaYwxpoqqAV1VdwBfAZYDLwL3q+oLInKjiMzOr7Yc2Coi64EVwFxV3ZpUoY0xxuxNVGNJZdesvb1dOzo6UnlvY4wJlYisUtX2Us8FPVPUGGPMHhbQjTEmIyygG2NMRlhAN8aYjLCAbowxGRH09dDT4MMNNXwogzHGPxbQa+DDDTV8KIMxxk+DIqDHVaOtdEONRgVTH8pg0mEtM1NN5gN6nDVaH26o4UMZTONZy8y4yHynaJy3qSt344xG3lDDhzKYxrPbLRoXmQ/ocdZofbihhg9lMI1nLTPjIvMBPc4arQ831PChDKbxrGVmXGQ+hz535rgBuUeor0brww01fCiDaay492OTTZkP6P2BL+ujA2wERLYNlv24mO3XtbHL52ZA8QgIyNXeilMxdnCYkLju14NNpcvnWkDPgOkLHqOrROdYkwi7VDm8tYXTjxnJL1Z12cFhglFuv25rbeEP8z6RQon8UCmgZz7lUk6WaqvlRjrszJ+su3p6uffpNyk+dduEJOMzG9lTu8yPcimlvynX1dOLsmeSxtI1xfe+DoPLSIdy7TA7OIyvbGRP7QZlQM/aJI1SY9Nd2cFhfGVzLmo3KFMuWWvKFY+A2Edkd7qlkDCwpm4Hh/HZYB3ZU49BGdAPb20p2dkScm21cGx6udEBnzmxjRUvdVc8OLLUt2DCZ3MuauMU0EVkFvBDoAm4XVUXlFnvM8ADwFRV9XYIS9YnaUSt2dgFoIwJW9WALiJNwC3AJ4FO4BkRWaaq64vWGwpcDfwxiYLGaTA05aLUbOzSvMaEzaWGfhLwiqpuBBCR+4BzgfVF6/0D8F1gbqwlTEipgDfY0w1Z61swZrBxGeXSBmwqeNyZX7abiJwAjFbVhyu9kIhcKSIdItLR3d1dc2GTlLWhjFHYMDETxdI1XUxf8Bhj5z3M9AWPDapjxjd1d4qKyD7AD4BLq62rqouBxZCbKVrve8epnnRDqZp9/2uGVNvPet+CiZ/1u/jFJaB3AaMLHo/KL+s3FJgAPC4iAB8BlonIbJ87RotFTTeU2qHn/nwdCPTt3DNTM4SdfDD0LZh4Wb+LX1wC+jPA0SIyllwgvwj4fP+TqroNOKT/sYg8DvyPkII5RB/KWGqH7tu1d+MjlJ3chomZWtTT7zLY+6ySUDWHrqo7gK8Ay4EXgftV9QURuVFEZiddwEaJOiutlg5D61w0WRO138X6rJLhNPVfVR9R1Y+q6n9S1f+VX3a9qi4rse5podXOofydgICKHT61dBha56LJmqgVoaxdfsMXg3KmaDnF6QaXDp9SHYnN+8iAHDpY56LJpqj9LjZENhkW0Ctw6fApt0OXWmb5QZNFUfpdsnj5DR9YQK/AtRZRboe2AG5MaTZENhkW0CuwWkR5NkLB1MOGyCbDAnoFVosoLY3JJHYCyR4bIhs/C+gVWC2itEZPJrHZiPUZjCfDwbjNYAG9KqtF7H1wlEpDQXIjFGw2YnS1nAyzEgQHcwVgUN6CLg2hXsCo1AQQKbNuUn0LNsQtOtfx3lma6DOYx7hbDb0BQq4xlDo4lMbezs61czorNcw4uZ4Ms9QKakQFwNd9zWroDRByjaHcQaCw16zapHZol9mIWaphxsl1an6WWkFJXwba533NAnoDhHywlDsI2lpb+MO8T/Dagk/xh3mfSLR2Uu6yDIXvGfJJM0muU/OzdC38qJcjcOXzvmYplwYIeTy7L0M3q3VOx33S9LVJXSvXkVq+fM9xqDR7e/qCx+r+Tn2uoFlAb4CQD5ZQhm7GedIMuc+jFJeRWqW+59OPGcnC5Rv4xpK13n7v5US5LpMrnytooprOjYPa29u1oyO4izJGlpUan6+KD1jInTSj5PanL3is5AHbn2YaDOL8PH3g+p26HKdpfzYiskpV20s9ZzX0BrHx7MmKsyXhc5O6UbI06gXcvlPXWrzPrVYL6INclloOrifNatucRpPat++h3pNa1O1J6nNw+U5rOYn5WkGzUS6DmK/Dr5KchOWyzUmPkohSpkarZ9RL1O1J8nNw+U6z0DKzgD6IpTH8qlqwTjq4uWyz692rvr30uVhOPD4Og6vnpBZ1e5L8HFyGvmZh6KalXAaxRtdISuUo5/58HX//zy/Qs72Pw1tb2P7BjkRzt1GvcV+q7D95+s3dz/efeDre+AsrXuoO/u499eSJo25P0p9DtTRJyKPR+llAH8QanSsuVQPr26W8s70PoOxFvyC+gzrqNpcqe7Hevp3c+/Sbuy+J4Do0rp7vIcnce9Q8cdTtSfsSDz53drqylMsgFmeu2CXvXU9QjuskE3WbXctePAjYJWUQtUz1pKeS7KeIuj0+XOJhzpS2hs2AToJTQBeRWSKyQUReEZF5JZ7/poisF5FnReRRETky/qKauLnkFV24HmRRg3Kczd6o21zPCaXaySBqmaLmnBsRFKNsj13ioX5VJxaJSBPwJ+CTQCfwDHCxqq4vWOd04I+qul1E/htwmqpeWOl1B9vEoiyrZdJGcY6ylNaWZg7cf1+vmr0uZS++AmW/pCYkjZ33cMn3E+C1BZ8q+38hT5yKus3l+DZc1EW9E4tOAl5R1Y35F7sPOBfYHdBVdUXB+k8DX4heXBOaWjoaYU+OclhLM3/9YAd9O/ccoi3NTcyfPd67g6rc1PjCDtDTjxnJL1Z1NaxTLWqu2sdOWFd2iYfKXAJ6G7Cp4HEn8LEK618G/LrUEyJyJXAlwBFHHOFYROO7Wg6yUqNHQqkhuXQSth/5oYZtT9RRGT5fi6SaOEeiZG02LMQ8ykVEvgC0A6eWel5VFwOLIZdyifO9TXrKHWSnHzOy6tXt6plx5+PJwGV74ip31FEZIQ/Ps0s8VOYS0AJHd9oAAAZNSURBVLuA0QWPR+WXDSAiZwDfAk5V1f+Ip3gmBOXSEYXph7ibs67NZR+CfmEZitNM9X4uUU6I9QRFHz7PuKbdh9xSKcelU3Rfcp2iM8gF8meAz6vqCwXrTAEeAGap6ssub2ydotmWdMeby+unfVW8cmUoJYQOSR8+zziFuj11dYqq6g4R+QqwHGgC7lDVF0TkRqBDVZcBC4GDgJ+LCMCbqjo7ti0wwUm6Oevy+j7kSF0mJEEY97us5/NMumYf5fWTnkiURmvGKYeuqo8AjxQtu77g7zNiLpcJXNLNWZfX9yFH6vpecd/vMolUV9TPM+nRJPW8flJXTUxrBI3NFDWJSPqKhS6v73qxpSRnTboE6lDudxn14lVJTwbycbJRWmWygG4SEdcs1Hpe34ep5KXK0LyPMPyA5kQ+lyRbJXFfNqGR6bd6RDnhp9U6tItzmcQkfROAaq/vkiNNOs/e6As+JZnqirotPqTfooo6mqr1gObdF52Lu0yVWEA3mVYt6DeiJhX1xBalUy3pMeZRtiXpMiX5+i4n/FJBv3kfoblJ9poFnfRYfwvoZlDzdSxy1E41Hy8Bm3SZknz9qKOp+nZpKtcksoBuBjVfZ03Wkwry8X6XaaffoqpnNNW23j7W3nBm7GWqxDpFzaCWdOdtVD4MuTTxjqZqBKuhm0HPxxqtr6mgwcYlneNTK88CujEe8ilIDHZxjKZqFAvoxnjIpyBhqvOllWcB3RhP+RIkTDisU9QYYzLCAroxxmSEBXRjjMkIC+jGGJMRFtCNMSYjqt6CLrE3FukG3oj474cAW2IsTiNZ2dNhZU9HqGX3udxHqurIUk+kFtDrISId5e6p5zsrezqs7OkIteyhlttSLsYYkxEW0I0xJiNCDeiL0y5AHazs6bCypyPUsgdZ7iBz6MYYY/YWag3dGGNMEQvoxhiTEcEFdBGZJSIbROQVEZmXdnkqEZE7RGSziDxfsOxDIvIvIvJy/vfwNMtYjoiMFpEVIrJeRF4Qkavzy70uv4gMEZF/E5F1+XL/fX75WBH5Y36/WSIi+6Vd1nJEpElE1ojIr/KPgyi7iLwuIs+JyFoR6cgv83p/6ScirSLygIi8JCIvisjHQyl7oaACuog0AbcAZwHHAReLyHHplqqiHwOzipbNAx5V1aOBR/OPfbQD+FtVPQ6YBlyV/6x9L/9/AJ9Q1eOBycAsEZkGfBe4WVX/M/AOcFmKZazmauDFgschlf10VZ1cMIbb9/2l3w+B36jqMcDx5D7/UMq+h6oG8wN8HFhe8Pha4Nq0y1WlzGOA5wsebwAOy/99GLAh7TI6bscvgU+GVH7gAGA18DFys/72LbUf+fQDjCIXPD4B/AqQgMr+OnBI0TLv9xdgGPAa+UEiIZW9+CeoGjrQBmwqeNyZXxaSQ1X17fzf/w4cmmZhXIjIGGAK8EcCKH8+ZbEW2Az8C/Aq0KOqO/Kr+Lzf/CPwP4Fd+ccjCKfsCvxWRFaJyJX5Zd7vL8BYoBu4M5/qul1EDiSMsg8QWkDPFM2d+r0eNyoiBwG/AL6uqv+v8Dlfy6+qO1V1Mrna7knAMSkXyYmIfBrYrKqr0i5LRH+jqieQS4leJSKnFD7p6/5C7s5tJwA/UtUpwF8pSq94XPYBQgvoXcDogsej8stC8mcROQwg/3tzyuUpS0SayQXze1X1wfziYMqvqj3ACnJpilYR6b/loq/7zXRgtoi8DtxHLu3yQ8IoO6ralf+9GXiI3Mk0hP2lE+hU1T/mHz9ALsCHUPYBQgvozwBH53v99wMuApalXKZaLQO+mP/7i+Ry094REQH+L/Ciqv6g4Cmvyy8iI0WkNf93C7m8/4vkAvtn86t5V24AVb1WVUep6hhy+/ZjqnoJAZRdRA4UkaH9fwNnAs/j+f4CoKr/DmwSkXH5RTOA9QRQ9r2kncSP0IFxNvAncnnRb6Vdnipl/RnwNtBHrhZwGbmc6KPAy8DvgA+lXc4yZf8bck3MZ4G1+Z+zfS8/MAlYky/388D1+eVHAf8GvAL8HNg/7bJW2Y7TgF+FUvZ8Gdflf17oPzZ9318Kyj8Z6MjvN0uB4aGUvfDHpv4bY0xGhJZyMcYYU4YFdGOMyQgL6MYYkxEW0I0xJiMsoBtjTEZYQDfGmIywgG6MMRnx/wGWg599eFq97AAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
   "source": [
    "f, ax = plt.subplots()\n",
    "ax.scatter(range(len(losses)), losses, label='train loss')\n",
    "ax.scatter(range(len(val_losses)), val_losses, label='val loss')\n",
    "plt.legend(loc='upper right');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KzGW3ztODYzV"
   },
   "source": [
    "The [ACNN paper](https://arxiv.org/pdf/1703.10603.pdf) showed a Pearson $R^2$ score of 0.912 and 0.448 for a random 80/20 split of the PDBbind core train/test sets. Here, we've used an 80/10/10 training/validation/test split and achieved similar performance for the training set (0.943). We can see from the performance on the training, validation, and test sets (and from the results in the paper) that the ACNN can learn chemical interactions from small training datasets, but struggles to generalize. Still, it is pretty amazing that we can train an `AtomicConvModel` with only a few lines of code and start predicting binding affinities!  \n",
    "From here, you can experiment with different hyperparameters, more challenging splits, and the \"refined\" set of PDBbind to see if you can reduce overfitting and come up with a more robust model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VcDLwf-20tci",
    "outputId": "35ae9353-5dd8-4397-bbf0-d58cfebb6584"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train {'pearson_r2_score': 0.9311347622675604\n",
      "val {'pearson_r2_score': 0.5162870575992874}\n",
      "test {'pearson_r2_score': 0.4756633065901693}\n"
     ]
    }
   ],
   "source": [
    "score = dc.metrics.Metric(dc.metrics.score_function.pearson_r2_score)\n",
    "for tvt, ds in zip(['train', 'val', 'test'], datasets):\n",
    "  print(tvt, acm.evaluate(ds, metrics=[score]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FrIO9CSgAHlz"
   },
   "source": [
    "### Further reading\n",
    "We have explored the ACNN architecture and used the PDBbind dataset to train an ACNN to predict protein-ligand binding energies. For more information, read the original paper that introduced ACNNs: Gomes, Joseph, et al. \"Atomic convolutional networks for predicting protein-ligand binding affinity.\" [arXiv preprint arXiv:1703.10603](https://arxiv.org/abs/1703.10603) (2017). There are many other methods and papers on predicting binding affinities. Here are a few interesting ones to check out: predictions using [only ligands or proteins](https://www.frontiersin.org/articles/10.3389/fphar.2020.00069/full), [molecular docking with deep learning](https://chemrxiv.org/articles/preprint/GNINA_1_0_Molecular_Docking_with_Deep_Learning/13578140), and [AtomNet](https://arxiv.org/abs/1510.02855)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wqS0gGXunB_s"
   },
   "source": [
    "# Congratulations! Time to join the Community!\n",
    "\n",
    "Congratulations on completing this tutorial notebook! If you enjoyed working through the tutorial, and want to continue working with DeepChem, we encourage you to finish the rest of the tutorials in this series. You can also help the DeepChem community in the following ways:\n",
    "\n",
    "## Star DeepChem on [GitHub](https://github.com/deepchem/deepchem)\n",
    "This helps build awareness of the DeepChem project and the tools for open source drug discovery that we're trying to build.\n",
    "\n",
    "## Join the DeepChem Gitter\n",
    "The DeepChem [Gitter](https://gitter.im/deepchem/Lobby) hosts a number of scientists, developers, and enthusiasts interested in deep learning for the life sciences. Join the conversation!"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "14_Modeling_Protein_Ligand_Interactions_With_Atomic_Convolutions.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
